Handling a Subset of a Large Dairy Industry Dataset for Quantitative Genetic Analyses of Extended Lactation Traits in Australian Dairy Cattle

Handling large-scale industry data is challenging both in the preliminary screening of data for validation, and also in the subsequent statistical analyses to obtain estimates of fixed effects, genetic parameters and estimated breeding values (EBVs). This paper summarises how a small subset from large industry-derived dairy data records have been explored and edited to be ready for genetic analysis prior to the analysis of a much larger data set, with a focus on lactation persistence and extended lactation traits in Australian dairy cattle. To cope with the large data volume (>158 million test-day records) test-day data were randomly divided into data-subsets, each of which could be effectively managed through a semi-automated quality control procedure for removal of extraneous outlier records. The methods as applied to one such data subset are reported here. The goals of this research were an investigation of extended lactation, but it was apparent that there were many instances of the calving date not being recorded, and hence initially being flagged as an extended lactation. Methods were derived for detecting potential “double lactations”, based on comparing singleand double-lactation curve models. These techniques are illustrated using test-day records from a sample of ~1 million cows recorded between 1985 and 2010 obtained from the Australian Dairy Herd Improvement Scheme (ADHIS).